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Abstract 

Balakrishnan and Mi [2] considered order preserving property of maxi- 
mum likelihood estimators. In this paper there are given conditions under 
which the moment estimators have the property of preserving stochastic 
orders. The property of preserving for usual stochastic order as well as for 
likelihood ratio order is considered. Sufficient conditions are established for 
some parametric families of distributions. 
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1. Introduction and preliminaries 

Suppose that X = (X\,X2, . . . ,X n ) is a sample from a population with density 
f(x; 9), where 9 £ C 1Z. We now consider the estimation of 9 by the method of 
moments (see, for example, Borovkov [1]). Let g : 1Z — > 1Z be a function such that 
the function m : — > 1Z 

/oo 
g(x)f{x] 9)dx 
-oo 

is monotone and continuous on 0. Particularly, if m is strictly monotone, then m 
is one-to-one and m~ l exits. Then for every t £ m(0) exists a unique solution of 
the equation m(9) = t, i.e. 9 = m _1 (t). In the case when m is nondecreasing or 
nonincreasing we can take m _1 (t) = inf{# : m{9) < t}, for any t £ m(0). Let 



s=4 

i=l 

be the generalized empirical moment based on X. 
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If g G m(B), then the estimator obtained by the method of moments (to be 
short moment estimator) is of the form 

(1.1) e = m-\g). 

We know that these estimators are strongly consistent. We often put 
g(x) = x k , k > 1. Then g reduces to - fcth empirical moment, i.e. = 
- Yu7=i -^-i- Particularly, m 1 = X. 

In this paper our aim is to give conditions under which the moment estimators 
have the property of preserving stochastic orders. We shall deal with stochastic 
orders, so recall their definitions. 

Let X and Y be two random variables, F and G their respective distribution 
functions and / and g their respective density functions, if they exist. We say that 
X is stochastically smaller than Y (denoted by X < st Y) if F(x) > G(x) for all 
x G TZ. The stronger order than usual stochastic order is likelihood ratio order. We 
say that X is smaller than Y in the likelihood ratio order (denoted by X <\ r Y) 
if g(x)/f(x) is increasing in x. Let F^ 1 and G~ l be quantile functions of F and 
G respectively. We say that X is less dispersed than Y (denoted X <di S p Y) if 
G~ l {a) — F~ l (a) is an increasing function in a G (0, 1). The family of distributions 
{f(x;9),9 G B C TZ} is stochastically increasing in 9 if X{9{) < st Y{9 2 ) for all 
9\ < &2 G ©, where X(6) has a density f(x;9). For example, it is well known 
that location parameter family {f(x — 9),$ G R} and scale parameter family 
{l/9f(x/9),9 > 0}, x > 0, are stochastically increasing in 9. For further details 
on stochastic orders we refer to Shaked and Shanthikumar [8] and Marshall and 
Olkin [6J. 

Furthermore, we say that the family of distributions {/(x; 9), 9 G C 1Z} has 
monotone likelihood ratio if f (x; $2) / f (x; 9i) is increasing function in 9 for any 
9\ < #2 ^ @, i-e. X(9i) <i r X(02)- For example, the following families have the 
monotone likelihood ratio in 9: 

(a) the family of distributions with density f(x; 9) = c(6 l )/?,(a;)/(_ OOi 0)(a;); 

(b) the family of distributions with density f(x;9) = c(9)h(x) exp(r](9)t(x)); 
provided that both 77 and t are increasing. 

The likelihood ratio order is closely related with total positivity, (see Karlin 
[3]). Let k(x,y) be a measurable function defined on X x Y, where X and Y are 
subsets of 1Z. We say that k(x, y) is totally positive of order r (to be short k(x, y) 
is TP r ) if for all x\ < ■ ■ ■ < x m , Xi G X; and for all y± < ■ ■ ■ < y m , yi G Y; and all 
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1 < m < r, we have 



k{xi,y n 



k{x m , y Tl 



> 0. 



It is clear, that the ordering X{9{) <i r X{9 2 ) whenever 9\ < 9 2 G is equiva- 
lent that the density f(x; 9) is TP 2 . We refer to Karlin [5] for proofs of basic facts 
in the theory of total positivity. 

From the definition immediately follows that if g and h are nonnegative func- 
tions and k(x,y) is TP r then g(x)h(y)k(x,y) is also TP r . Similarly, if g and h are 
increasing functions and k(x,y) is TP r , then k(g(x), h(y)) is again TP r . 

Twice differentiable positive function f(x; 9) is TP 2 if and only if 



d 2 



dxd9 



log/(x;0)>O. 



Total positivity of many functions that arise in statistics follows from the basic 
composition formula. If g{x,z) is TP m , h(z,y) is TP n and if the convolution 



k(x,y) 



g(x,z)h(z,y)dz 



is finite, then k(x,y) is TP min ^ n ). 

A particular and important case is when k(x,y) = f(y — x). A nonnegative 
function / is said to be PF^ (Polya frequency function of order k) if f(y — x) is 
TP k . 

Recall that a real valued function / is said to be logconcave on interval A if 
f(x) > and log / is an extended real valued concave function (we put logO = 
— oo). It is well known (see, for example, Schoenberg |9J) that the function / 
is PF 2 if and only if / is nonnegative and logconcave on 1Z. Recall also a very 
important property of PF 2 functions which we will use in the sequel that if g and 
h are logconcave functions on 71, such that the convolution 



h(x) 



g(x — z)h(z)dz 



is defined for all x G 71, then the function h is also logconcave on 7Z. 

Another very important property of TP functions is the variation diminishing 
property. 

Lemma 1. Let g be given by absolutely convergent integral 

POD 

g(x) = / k(x,y)f(y)dy 



3 



where k(x,y) is TP r and f change sign at most j < r — 1 times. Then g changes 
sign at most j times. Moreover, if g changes sign j times, then f and g have the 
same arrangement of signs. 

The basic tool for proving stochastic ordering of estimators are order pre- 
serving properties of underlying stochastic orders, for the proof see Shaked and 
Shanthikumar [8j. 

Lemma 2. Assume that g is an increasing function. 

(a) IfX< st Y, theng(X) < st g(Y). 

(b) IfX< h Y, theng(X) < h g(Y). 

Lemma 3. Let X\, . . . , X n and Y\, . . . , Y n be independent random variables. 

(a) IfXi < st Y h i = 1, . . . , n, then X x + • • • + X n < st Y 1 + • • • + Y n . 

(b) If Xi <i r Yi, i = 1, . . . ,n, then X\ + ■ ■ • + X n <\ T Y\ H h Y n , provided 

these random variables have logconcave densities. 

Let 9 be a moment estimator of 9 based on a sample from population with 
density f(x; 9), 9 E © C 1Z. Let now 9\ and §2 be the moment estimators obtained 
on the basis of the sample X from population with density f(x\9{) and of the 
sample Y from population with density f(x; #2) respectively, where 9\ < 9 2 G ©. 
We say, that 9 is stochastically increasing in 9 if the family of its distributions is 
stochastically increasing in 9, i.e. 9\ < st 9 2 . We also shall be interested whether the 
stronger property holds 9\ <\ T §2, i.e. 9 is increasing in 9 with respect to likelihood 
ratio order. 

Note, that in general the moment estimator may not be stochastically mono- 
tone as the following example indicates. 

Example 1. Consider a sample X = (X±, . . . , X n ) from the uniform distribution 
on the interval (—9, 9), 9 > 0. Then E(X) = and the first moment contains no 
information about 9. We use the second moment. Then we have E(X 2 ) = 9 2 /3 = 
m(9). Thus the moment estimator of 9 is of the form 9 = \/3/n XlILi Since 
the random variable X is not stochastically increasing in 9 and m(9) is increasing 
for 9 > 0, the estimator 9 is not stochastically monotone. 

2. Results 

Assume for simplicity that for a given function g the generalized empirical moment 
belongs to m(G), the domain of the values m(9), 9 e 0, for every resulting set of 
observations. 
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The following theorem gives a sufficient conditions for likelihood ordering of 
moment estimators for one-parameter family in the case when the estimator is 
based on the sample mean. 

Theorem 1. Assume that the function m{9) = f^° oo xf(x;9)dx < oo for all 9, 
f(x;9) is TP 2 and f(x;9) is logconcave in x. Then the moment estimator 9 is 
increasing in 9 with respect to likelihood ratio order. 

Proof. Since f(x; 9) is TP 2 we have ordering X^) < lr X;(0 2 ) for all 9 1 < 9 2 e Q 
and i = l,...,n. From the variation diminishing property (see Lemma [T]) we 
deduce that function 

/oo 
(x — c)f(x; 9)dx 
-oo 

changes sign at most one for any c, from + to — if occurs. This implies that 
m(9) is increasing. Hence the moment estimator is of the form 9 = m^ 1 (X). Since 
/ is logconcave, we conclude from Lemma |3] (b) that X is increasing in 9 with 
respect to likelihood ratio order. Theorem follows from the preserving property of 
monotone likelihood order under increasing operations. □ 

If we weakness the assumptions of Theorem [T] we can obtain the following 
theorem. 

Theorem 2. Assume that the function m{9) = J™ xf(x; 9)dx < oo for all 9 and 
f(x; 9) is TP 2 . Then the moment estimator of 9 is stochastically increasing in 9. 

In the general case we can formulate the following corollary. We omit the proof 
because it follows by the same method as in Theorem [TJ 

Corollary 1. Assume that the function g is increasing and the integral m(9) = 
j_ g(x)f(x; 9)dx < oo for all 9. If f(x; 9) is TP 2 , then the moment estimator 9 
of the form U.l\) is stochastically increasing in 9. Moreover, if the random variable 
g{X) has logconcave density, then the estimator 9 is increasing in 9 with respect 
to monotone likelihood ratio order. 

The next Corollary follows immediately from Corollary [1] and form Proposi- 
tion 2 due to An [lj. 

Corollary 2. Let X be a random variable with a density f(x; 9) and g be a func- 
tion such the integral m{9) = J^° oQ g(x)f(x;9)dx < oo for all 9. Assume that the 
following conditions are satisfied: 
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(a) g is strictly increasing, concave and differentiable; 

(b) \-jkv (x)\ is logconcave, where v = g^ 1 ; 

(c) / is logconcave, decreasing on the support of X; 

(d) f(x;9) isTP 2 . 

Then the moment estimator 9 is increasing in 9 with respect to likelihood ratio 
order. 

In many situations we deal with one-parameter exponential family with den- 
sities of the form 

(2.1) f(x; 9) = h(x)c(9) exp(7](9)T(x)), 9 G 6, x G (a, b). 

Recall the well known formula for moments of T(X), see Berger and Casella [3J: 

(2.2) E $ (T(X)) = -^° g,i 



(2.3) Var e (T { X)) = ^W' + E ^ 



MO)}') 2 

It is easy to prove that if both T and 77 are increasing (decreasing), then 
f(x; 9) is TP2 and from the variation diminishing property it follows that m{9) = 
Eg(T(X)) is increasing (decreasing). Combining those facts with order preserving 
properties of the usual stochastic order we can formulate the following theorem. 

Theorem 3. For the one-parameter exponential family with densities of the form 
\2. where both 77 and T are increasing (decreasing), the moment estimator 9 = 
m~ l (l/n'Y^ =1 T(Xi)) is stochastically increasing in 9. 

Let us make the following observations. Now we consider the maximum 
likelihood estimation of 9 for the one-parameter exponential family (12. ip . Let 
x = \pc\^ . . . , x^ be a value of a sample X. The maximum likelihood function is 
of the form 

L(x; 9) = f[ h( Xi ) [c(9)} n exp L(9) £ T 

So we have 

aiogL(x;g) =n[logc(g)]/ + [ ^ )r ^ T( ^ 

i=l 

It is clear that ' — - = if and only if 

89 




n 

(2.4) r5>(*i) 



[logc(fl)]' 
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Let 9 be a solution of the equation (12. 4ft . Then 9 is the maximum likelihood 
estimator (MLE) since using (12. 3p we have 



d 2 logL(x; 



d9 2 

On the other hand 9 is the moment estimator, since E0(T(X)) 



n([logc(9)}") 2 \ e=§ -Var § (T(X))<0. 

Pogc(0)]' 



fo(*)]' ' 

Thus from Theorem [3] we have the following result. 

Theorem 4. For the one-parameter exponential family with densities of the form 
\2. where both rj and T are increasing (decreasing), the maximum likelihood 
estimator 9 is stochastically increasing in 9. 

Particularly, Theorems 2 and 3 of Balakrishnan and Mi [2] give conditions 
under which maximum likelihood estimators for the one-parameter exponential 
family of the form (12. ip are stochastically increasing. In the Theorem H] we have 
proved it but under weaker assumptions. 

Example 2. Let X = . . . , X n ) be a sample from the distribution with density 



f(x;6) = d^exp(-9/x), x > 0, 9 > 0. 

This is one-parameter exponential family with T(x) = 1/x and rj(9) = —9. Using 
(12. 2p we get moment estimator 9 = (2/n ^]™ =1 l/Xj) -1 . By Theorem [3] the esti- 
mator 9 is stochastically increasing in 9. Of course, 9 is also maximum likelihood 
estimator. 

Example 3. Let X = (X±, . . . ,X n ) be a sample from the gamma distribution 
with density 

A ) = w L ^" 1 exp(-x/A), x > 0, A > 0, 
r(a)A a 

where a > is known. This is clearly exponential family with T(x) = x and 
77(A) = — 1/A, so 

77a v 

By Theorem [1] the estimator A = X/a is stochastically increasing in A. Moreover, 
if a > 1, then the density / is logconcave and by Theorem [TJ the estimator A for 
a > 1 is also increasing with respect to likelihood ratio order. 



|-(-alogA-r(a)) 
E X (X) = ^ A- ^ = aX. 
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On the other hand, assume now that A is fixed and a is unknown. This is also 
one-parameter exponential family with T{x) = logx and f]{a) = a. Using (12. 2\i 

we get 

d 

E a (\og(X)) = — (bgr(a) + a log A) = *(a) + log A = m(a), 
oa 

where ^(a) = ^ logT(a) is the digamma function. By Theorem [3] the estimator 
a = m~ l (T) stochastically increasing in a. This estimator obtained by the method 
of maximum likelihood was considered in Example 2 of Balakrishnan and Mi 

Example 4. Let X = (Xi,...,X n ) be a sample from the logistic distribution 
with density 

n/ „s 9exr»(—x) ^ „ 

/M =77- — V ,L +iy xen,e>o. 

(1 + exp (— x))^ +i > 

This is the one-parameter exponential family with T(x) = log(l + exp(— x)) and 
rj(6) = 9. From (12. 2p we get Eg(T(X)) = 1/9, hence from Theorem[T]the estimator 
9 = 1/T is stochastically increasing in 9. 

Example 5. Let X = . . . ,X n ) be a sample from the uniform distribution 
on the interval {0,9), 9 > 0. Them moment estimator of 9 based on the first 
empirical moment is of the form 9 = 2X and by Theorem [1] this estimator is 
increasing in 9 with respect to likelihood ratio order. Consider another moment 
estimator based on generalized moment g = J^™ =1 logXj. Easy calculations 
show, that Eg(g(X)) = log# — 1, where g(x) = logx, thus from Corollary [2] 
the moment estimator 9 = exp(l/n J^™ =1 logXj — 1) is also increasing in 9 with 
respect to likelihood ratio order. 

Now we consider the case when our family of distribution is the location family. 
Then we can formulate the following theorem. 

Theorem 5. Let T = {f(x — 9), 9 G 1Z} be the location family. Suppose that \i-y = 
E(X(0)) = j'^' oo xf{x)dx < oo. Then the estimator 9 = X — /ii is stochastically 
increasing in 9. Moreover, if f is logconcave then 9 is also increasing in 9 with 
respect to likelihood ratio order. 

Proof. The estimator 9 is stochastically increasing in 9 since the family J 7 is 
stochastically increasing in 9. If we assume that / is logconcave, i.e. f(x — 9) is 
TP2, then the family J 7 has monotone likelihood ratio. Since the convolution of 
logconcave functions is the logconcave function we deduce that the estimator 9 is 
also increasing with respect to likelihood ratio order. □ 
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Similar results we may obtain for a scale parameter family. The ordering prop- 
erty of the moment estimators of 9 in this case is described by the following 
theorem. 

Theorem 6. Let T = {l/6f(x/6),6 G R+}, x > 0, be the scale parameter family. 
Suppose that exists k-th moment of the random variable X(l) with density f(x) 
and E(X k (\)) = Then the moment estimator 9 = v/m&7 A*fe ^ s stochastically 
increasing in 9. Moreover, if Var(X(l)) = a 2 < oo, where S 2 = ni2 — m\ is the 
sample variance, then the another moment estimator for 9 given by 9 = y/S^/a, 
is also stochastically increasing in 9. 

Proof. The first part of theorem is obvious since the family T is stochastically 
increasing in 9. It is also easy to prove that if Q\ < 92 G 0, then X(9\) <di S p Xfa) 
and then the vector of spacings U = (Ui, . . . ,U n ), where [/, = X i:n — JQ_i :n , 
% — 1, . . . , n (we put Xo :n = 0), is stochastically increasing in 9 (see Oja [7]). Since 

S 2 = ^ £ (X (J) - X (t) ) 2 = ^ Yl (Uj + U^ + .-. + U^f, 

l<i<j<n X<i<j<n 

then the sample variance is an increasing function of vector U. Thus the theorem 
follows from the stochastic preserving property of multivariate stochastic ordered 
vectors under monotone operations. □ 

Example 6. Let X = (Xi,...,X n ) be a sample from the logistic distribution 
with density 

fix; 9) = ^PH g ~ 6 » x e K, 9 G K. 
1 + exp(— [x — 9)) 

It is not difficult to see that density / is logconcave and Eg(X) = 9. By Theorem [5] 
the estimator 9 = X is increasing in 9 with respect to likelihood ratio order. 

Example 7. Let X = (X 1; . . . ,X n ) be a sample from the Weibull distribution 
with density 

f{x; 9) = (l/9)x 1/e - 1 exp (-x 1/e ), x > 0, 9 > 0. 

It is obvious that the family of these distributions is not stochastically ordered 
in the parameter 9. Let Tj = — logJQ, i = l,...,n. After easy calculation we 
have that T\ = st 9W\, where W\ is the Gumbel distribution with cumulative 
distribution function e~ e x , x G 1Z. Thus the distribution of T\ belongs to the 
scale parameter family. It is known that E{W\) = 7, where 7 is Euler constant and 
VariyVi) = 7r 2 /6, hence 9 = ^6S 2 /n and 9 = T/7 are moment estimators for 9, 
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where Sj< = l/ n SILi(^i — T) 2 , but we can not apply here Theorem [6] since the 
support of W\ is not TZ + . So, let us consider Zi = | log JQ|, i = 1, . . . ,n. Then we 
have Z\ = st 0|Wi| and \x = E[Z\) = 7 — 2Ei(— 1) = 1.01598 approximately, where 
Ei(x) = — J m e - ' jtdt. Also, after calculations we have o 2 = Var{Z\) = n 2 /6 + 
4(7 — Ei(—l))Ei(—l) = 0.945889 approximately. By Theorem O the estimator 
9 = \f^zl a i s stochastically increasing in 9. The same is true for the estimator 
9 = f/fi. 
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